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What^s Known 


This report summarizes the research on the association between state interventions 
in chronically low-performing schools and student achievement. Most of the research 
focused on one type of state intervention: working with a turnaround partner. Few 
studies were identified that examined other types of interventions, such as school 
closure, charter conversion, and school redesign. Most studies were descriptive, which 
limits the conclusions that can be drawn about the effectiveness of the interventions. 
Results of studies of turnaround partner interventions were mixed and suggested that 
student achievement was more likely to improve when particular factors — such as strong 
leadership, use of data to guide instruction, and a positive school culture characterized 
by trust and increased expectations for students — were in place in schools. 


Current federal education policies promote a substantial role for state education agencies^ in school 
improvement. However, administrators in a majority of state education agencies report that improving 
their lowest performing schools is challenging, perhaps because the work is complex and research on 
effective strategies is lacking (Tannenbaum et ah, 2015). Federal initiatives such as the No Child Left 
Behind Act, Elementary and Secondary Education Act Flexibility, School Improvement Grants, and the 
Every Student Succeeds Act provide guidance to state education agencies on intervening in chronically 
low-performing schools, but because of differences in state policies, the ways that state education agen- 
cies are able to intervene vary widely (Klute, Welp, Yanoski, Mason, & Reale, in press). Furthermore, 
state policies related to school improvement frequently change. For example, in 2014 nearly a quarter of 
states considered new legislation related to school improvement or related to sanctions or interventions 
for chronically low-performing schools (Education Commission of the States, 2015). 
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Three states in the Regional Educational Laboratory (REL) Central Region — Colorado, Missouri, and 
Nebraska — were among those considering legislation related to school improvement, sanctions, or inter- 
ventions for low-performing schools in 2014- REL Central Governing Board members indicated that one 
of the most pressing issues they were facing was how to improve chronically low-performing schools. State 
education leaders in the region expressed a need to gain a national perspective on the range of ways that 
states are currently intervening with their lowest performing schools and what research suggests about 
the effectiveness of different approaches. They indicated that such information would be helpful as they 
considered new approaches in their own states. This report is related to another REL Central study (Klute 
et al., in press) that summarizes state policies for intervening in chronically low-performing schools. 

What the study examined 


As state policymakers and education leaders consider ways that state education agencies can support the 
lowest performing schools, they may benefit from awareness of the research on the association between 
state interventions in chronically low-performing schools and student achievement. This report thus sum- 
marizes research published from 1994 to 2014 to address the question, “What does research suggest about 
the interventions and implementation features associated with improving student achievement in chron- 
ically low-performing schools?” 

Lour electronic databases were searched to identify relevant research studies on the association between 
state interventions in chronically low-performing schools and student achievement (see box 1 for defi- 
nitions of these terms). The study team searched for research on all the ways that states intervened to 
improve chronically low-performing schools, whether specifically tied to a federal initiative or not. The 25 
studies described in this report met the study eligibility criteria of fitting the definition of relevant research, 
focusing on a state intervention including chronically low-performing K-12 schools conducted in the 
United States, and focusing on student outcomes (see appendix A for a description of the screening and 
review processes and appendix B for characteristics of the 25 studies). The studies used three main types of 
research designs, each of which has inherent limitations (box 2). 


Box 1. Key terms 

Chronically low-performing schools. Schools classified by states as among the lowest performing schools in 
the state. This category includes schools classified as "priority schools” in states with approved Elementary 
and Secondary Education Act Flexibility Requests, schools planning for or in restructuring under No Child Left 
Behind because they failed to make adequate yearly progress for five or more years, schools receiving School 
Improvement Grants because they were among a state’s persistently lowest achieving schools, and schools 
classified as among the lowest performing schools through state accountability or accreditation policies. Within 
the parameters of the federal initiatives, states vary in how they define their lowest performing schools (for 
example, Dillon & Rotherham, 2007; Perlman, 2013). 

Relevant research studies. Studies with stated research questions that linked a state intervention to student 
academic outcomes and that clearly described research methods. 

State interventions. Actions taken directly by states or their designees, or actions that states require schools or 
districts to take, to improve chronically low-performing schools. This report focuses on interventions with individ- 
ual chronically low-performing schools rather than on districtwide interventions. Interventions did not have to be 
tied to a specific federal initiative to be included. 
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Box 2. Research designs used by studies inciuded in this report 

Descriptive study. A study with no comparison group. These studies often use qualitative methods and provide 
rich detail about the implementation of an Intervention in one school or a small group of schools. Because they 
do not compare the schools under study with schools that did not receive the intervention, it is impossible to 
attribute any observed changes In student outcomes to the intervention Itself. This type of study is useful for 
identifying factors that should be examined in future research using a well conducted impact study design. 

Quasi-experimentai design study. A study with a comparison group that did not receive the intervention. The 
studies use a nonrandom process to assign schools to the intervention or comparison groups. Well conducted 
quasi-experimentai designs can provide information about the impact of a program. The What Works Clearing- 
house standards for determining whether a quasi-experimentai design study is well conducted require a study 
to demonstrate that the two groups were equivalent on the outcome measure before the intervention, to use 
reliable and valid measures of outcomes, and to be free of confounds (What Works Clearinghouse, 2014). Con- 
founds are present when only one school Is assigned to each condition or when another factor is completely 
aligned with group assignment (for example, all the schools in the treatment group are in rural areas, while all 
the schools in the control group are in urban areas). 

Retrospective study. A study that compares schools that improved under the intervention to those that did not. 
These studies use a design that compares two groups that are formed based on student outcomes at the end 
of the intervention period. In this type of study, researchers look retrospectively at data that were collected 
during implementation of the intervention to identify factors that may explain the differences in outcomes for 
the two groups. Because of their retrospective nature, these designs cannot be used to provide causal evidence 
for the association of identified factors and outcomes. Furthermore, it is possible that any observed difference 
in outcomes between the groups is due to factors not examined by the study. This type of study is useful for 
identifying factors that should be examined in future research using a well conducted impact study design. 


What the study found 


Nearly three-quarters (18) of the 25 identified studies examined one type of state intervention in which 
the state required the school to work with a turnaround partner (table 1). Although different states refer to 
these partners with different terms, in all these interventions a technical assistance provider works with a 
school to identify the school’s needs, develop a school improvement plan, and provide technical assistance 


Table 1. Number of studies identified, by intervention type and research design 



Type of intervention studied 




Schooi 


Change 




improvement 


in entity 

Total 


Turnaround 

pianning 

Schooi 

operating Schooi 

number of 

Research design 

partner 

with funding 

redesign 

the schooi closure 

studies 

Quasi-experimentai design studies 
(studies with a comparison group that 
did not receive the intervention) 

6 

1 

0 

0 1 

8 

Retrospective studies (studies 
that compared schools that made 
improvements under the intervention to 
those that did not) 

3 

1 

0 

0 0 

4 

Descriptive studies (studies with no 
comparison group) 

9 

0 

2 

2 0 

13 

Total 

18 

2 

2 

2 1 

25 

Source: Authors' compilation. 
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in implementing the plan. The remaining seven studies examined school redesign, conversion to a charter 
school, replacement of staff, transfer of school leadership to an education management organization, and 
school closure. Descriptions of the research on each of these types of interventions are presented below. 
Less than a third of identified studies used a quasLexperimental design that compared schools that received 
an intervention to schools that did not, about one-sixth of the studies used a retrospective design, and over 
half of studies were descriptive studies with no comparison group. 

Research on turnaround partners 

Eight of the eighteen studies that focused on a turnaround partner intervention were conducted in Cah 
ifornia. The remaining studies were conducted in Alabama, Illinois, Kentucky, Louisiana, New Jersey, 
New Mexico, North Carolina (two studies). South Carolina, and Texas. The six studies that investigated 
the effects of turnaround partner interventions using a comparison group found mixed results for student 
achievement. Three studies were retrospective and found that schools that improved had strong leadership, 
used data to guide instruction, had a positive school culture characterized by trust, and had increased 
expectations for students. 

Studies in California 

Studies in California examined the Immediate Intervention/Underperforming Schools Program or its sue- 
cessor, the High Priority School Grant Program. The Immediate Intervention/Underperforming Schools 
Program was established by California’s Public Schools Accountability Act of 1999. Schools with an Aca- 
demic Performance Index score in the bottom half of all schools statewide were eligible for the program. 
The High Priority School Grant Program was created by the California state legislature in 2001 to replace 
the Immediate Intervention/Underperforming Schools Program. Schools ranked in the bottom 10 percent 
statewide were eligible for the High Priority School Grant Program. While the programs differed slightly, 
both involved low-performing schools voluntarily working with an outside evaluator to identify barriers to 
student achievement and develop a multiyear action plan to address those barriers. Schools received addi- 
tional funding to support implementation of the plan and faced increased sanctions if they did not demon- 
strate progress (Bitter et al., 2005; Goe, 2006; Harr, Parrish, Socias, & Gubbins, 2007; Timar & Chyu, 2010; 
Woods, 2004). Schools that failed to improve and faced increased sanctions were required either to work 
more intensively with a turnaround partner (called a school assistance intervention team) for three years or 
to enter into an agreement (called a joint intervention agreement) to address corrective actions by the state 
within 18 months or face further sanctions (Davis, 2007). 

Of the eight studies conducted in California, three focused on the Immediate Intervention/Underperform- 
ing Schools Program in general, two focused on the High Priority School Grant Program in general, two 
focused on schools that had not made progress and had entered into a joint intervention agreement, and 
one focused on a school that had not made progress and had worked with a school assistance intervention 
team. 

Studies of the Immediate Intervention/Underperforming Schools Program. Two of the three studies of the 
Immediate Intervention/Underperforming Schools Program examined more than 2,000 schools and com- 
pared schools that participated in the program with similar schools that did not participate.^ Both Bitter 
et al. (2005) and Goe (2006) reported that student achievement was similar in participating schools and 
comparison schools before the intervention and remained similar after the intervention. 

An additional study examined more than 100 schools and compared participating schools that were 
deemed successful to participating schools that were less successful.^ Woods (2004) relied on interviews 
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with principals who identified factors that distinguished successful and less successful schools, including 
leaders (school board members and district and site administrators) who set a positive tone, well articulated 
curricula that address a clear progression of learning goals from one grade to the next, and systems for 
analyzing individual student progress and designing instruction to meet the needs of all students. Although 
the study attributed whether schools were successful to the factors, other factors not identified may explain 
the difference in outcomes. 

Studies of the High Priority School Grant Program. Results of two studies examining the High Priority School 
Grant Program echoed the findings of studies examining the Immediate Intervention/Underperforming 
Schools Program. 

Harr et al. (2007) examined data from more than 300 schools and compared schools that participated in 
the High Priority School Grant Program to schools that did not participate. The comparison group was the 
lowest performing schools that did not participate in the program, a group of schools that served “slightly 
less challenging populations” (p. 2) and that were slightly higher performing than schools that participated 
in the program. Participating schools demonstrated gains in student achievement after being selected for 
the program, but the gains were similar to those in comparison schools. The results should be interpreted 
with caution because of the differences between the two groups of schools before the intervention. In addh 
tion to the difference in student achievement, participating schools tended to serve a higher proportion 
of students with special needs than comparison schools, which also may have contributed to the lack of 
differences in outcomes that was observed. 

Timar and Chyu (2010) conducted a descriptive study of 15 schools that participated in the High Priority 
School Grant Program and compared schools that improved to schools that did not improve."^ Several 
factors distinguished schools that improved from schools that did not improve, including more trust among 
staff, more stability in staff, and stronger leadership. Schools that improved also used their additional 
funding and action plans differently from how other schools used them. Specifically, schools that improved 
tended to use the additional funding for purposes that were closely tied to the school’s vision for improve- 
ment, while schools that did not improve tended to use the funding to satisfy immediate needs, such as 
funding items that the school could not afford to pay for out of its regular budget but that were not tied in 
any way to the school improvement plan. In schools that improved the action plan tended to be a living 
document that was continually referred to, evaluated, and updated, while in schools that did not improve 
the action plan tended to be less coherent and not referred to after it was initially submitted. In addition, 
other factors not identified in the study may explain the difference in outcomes for the two groups of 
schools. 

Studies of joint intervention agreements. Two studies focused on the joint intervention agreement process 
during which the state conducts a scholastic audit of the school to identify corrective actions.^ The district 
then enters into an agreement with the state to take those corrective actions within 18 months (Bibian, 
2006; Davis, 2007). Both studies were descriptive studies of a single middle school that had demonstrated 
improvement in student achievement overall (Davis, 2007) or improvement in achievement among English 
learner students (Bibian, 2006). In addition to establishing a joint intervention agreement, both schools 
experienced turnover in leadership and staffing that were not required by the state intervention. Both 
studies examined a variety of data sources, including interviews, surveys, observations, and archival data, 
and identified several factors associated with increased achievement, including strong leadership, positive 
changes in the school culture, and targeted professional development. 

Study of a school assistance intervention team. One study examined an elementary school that was working 
with a school assistance intervention team. Tapia (2008) focused on one component of the team’s work: 
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training teachers in the instructional materials adopted by the state board of education. The school that 
worked with the team was compared to another school that appeared on a Similar Schools Report prepared 
by the California Department of Education. The schools were similar in terms of demographic characteris- 
tics and Academic Performance Index scores (California Department of Education, n.d.). The study found 
mixed results: the school that worked with the team made greater increases on California’s accountabilh 
ty measure, the Academic Performance Index, but the comparison school outperformed the school that 
worked with the team in the percentage of students scoring proficient or advanced on the state assessments 
in language arts and math.^ Because only one school was included in each group, it is impossible to know 
whether the results were due to the intervention or to other idiosyncratic features of one or both schools. 

Studies in other states 

Ten studies examined interventions in states other than California. Studies varied somewhat in the details 
of the interventions studied, including whether additional funding was provided to support implementation 
of reforms. Unlike interventions in California, studies in other states did not describe school participation 
in interventions as voluntary. 

Studies in two states examined interventions that were similar to those in California, in that extra funding 
was provided to schools to support the implementation of improvement efforts. Both studies were descrip- 
tive and did not include a comparison group: 

• Kentucky’s distinguished educator program was created by the Kentucky Education Reform Act 
of 1990. Wakelyn’s (2006) case study examined four high schools using qualitative methods. Dis- 
tinguished educators worked with schools to develop a schoolwide transformation plan. All four 
schools demonstrated improvement in student achievement. 

• Cruz’s (2007) case study examined four elementary schools in New Mexico that were required by 
the state’s public education department to work with a state-assigned liaison who provided tech- 
nical assistance and helped the school develop an action plan for improvement. The schools also 
received training on the Malcolm Baldridge Education Criteria for Performance Excellence, which 
were adapted from the Baldridge Criteria for business (originally developed by the National Insth 
tute of Standards and Technology) to provide a framework for school reform focused on perfon 
mance management for continuous improvement. Principals at two of the four schools believed 
that the intervention had a positive impact on student achievement. Three of the four schools 
demonstrated improvement in student test scores in both reading and math, although only one 
made adequate yearly progress. 

The eight remaining studies of turnaround partners did not mention extra funding to support school 
improvement efforts: 

• Scroggins (2012) examined Louisiana’s Distinguished Educators Program, which was created by 
state legislation in 1999 and involved the state assigning distinguished educators to work in some 
schools that were rated academically unacceptable. These educators were charged with working 
closely with staff at the school to identify strengths and weaknesses, identify strategies for improve- 
ment, and provide assistance in implementing those strategies. The study compared schools that 
received distinguished educators to those that did not and examined whether test scores in grades 
3-5 differed for the two groups. The study does not provide information on how the comparison 
schools were selected. The results indicated a significant difference for grade 4 only: schools with 
distinguished educators had a higher average assessment score in English language arts and in 
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math than schools without a distinguished educator. Differences were not significant for the other 
two grades. However, because no information was provided about how similar the two groups were 
before the intervention, the results should be interpreted with caution. 

• Thompson, Brown, Townsend, Henry, and Fortner (2011) compared students’ academic achieve- 
ment in turnaround schools (schools that scored below 60 percent for at least two years on North 
Carolina’s Performance Composite, which summarizes the percentage of end-of-grade or end-of 
course assessments that students passed) and schools that performed only slightly better.^ Between 
2006 and 2010 turnaround schools received technical assistance to develop required school 
improvement plans consistent with the state’s Framework for Action and professional development 
and school-level coaching aligned with the goals identified in the plan. At the high school level 
students’ academic achievement was significantly greater in turnaround schools than in compari- 
son schools. Graduation rates increased more in turnaround schools than in comparison schools, 
though the difference was not statistically significant. At the middle school level turnaround 
schools made greater improvement in student achievement in reading, though the results were 
not statistically significant. Although differences between the turnaround and comparison schools 
before the intervention were controlled for, the study did not provide detailed information on how 
much the groups differed at baseline. The findings about the outcomes associated with the inter- 
vention should thus be interpreted with caution. The study also sought to identify characteristics 
that distinguished turnaround schools that improved from those that did not. The study identified 
the following factors: changes in principals and staff; culture of trust, accountability, and high 
expectations for students; professional development and follow-up coaching focused on the most 
pressing problems that the schools were facing; supports for improved instruction such as use of 
formative assessment and assigning struggling students to the most effective teachers; and district- 
level support related to staffing, support for data analysis, and assistance for struggling students. 
Other factors beyond these differences might explain the difference in outcomes. 

• Dominguez, Nicholls, and Storandt (2006) examined 62 schools that worked with an external 
review team and compared schools that improved to reach satisfactory status under South Caroli- 
na’s accountability system with schools that did not. External review teams were a strategy intro- 
duced by South Carolina’s Education Accountability Act of 1998, which prompted a shift in the 
state’s technical assistance efforts from districts to individual schools. Teams work with low-per- 
forming schools to diagnose strengths and weaknesses and develop and implement school improve- 
ment plans that specify the support, services, and technical assistance the state would provide to 
the school. The schools that did not improve appeared to have “deeper or more systemic problems 
as evidenced by lack of procedures to support alignment with state academic standards, not using 
student data to inform curricula, and not engaging in planning based on research-supported prac- 
tices” (p. 9). The two groups of schools differed in the content of technical assistance provided. 
Schools that improved most often received technical assistance in parent and community involve- 
ment, best practices for homework centers, and help selecting instructional materials. Schools that 
did not improve most often received technical assistance in understanding education research, 
planning and assessing professional development, and stakeholder involvement in curriculum 
development. Other factors that the study did not identify could explain the difference in out- 
comes for the two groups of schools. 

• Jefferson (2006) examined the Alabama State Takeover Model, which is part of the implementa- 
tion of the Alabama Education Accountability Plan, in 10 schools. The model involved a team of 
state-appointed staff assuming leadership of the school for one year during which the team ana- 
lyzed three to five years of assessment data and developed a school improvement plan that included 


7 


a seven month plan for instructional delivery. During implementation of the seven month plan, 
the team provided feedback on lesson plans, observed classrooms, and modeled effective strategies. 
Seven of the ten schools experienced an increase in student achievement, one school experienced 
little change, and two schools experienced a decline in student achievement. About three-quarters 
of teachers and administrators who responded to a survey reported that the process helped improve 
student achievement. 

• Four studies examined similar interventions: support for school improvement planning provided by 
the state system of support in Illinois, a technical assistance team in Texas, Collaborative Assess- 
ment and Planning for Achievement in New Jersey, and an assistance team in North Carolina.® 
Each used case study methodology to describe characteristics of schools that improved after expe- 
riencing these interventions. These studies highlighted factors such as increased expectations of 
students and accountability for staff, use of student data to drive instruction, and use of varied 
effective teaching strategies to meet individual students’ needs that appeared to be associated 
with increases in student achievement after the state intervention (Cepela, 2008; Johnson, 2008; 
Macias, 2011; Sipp, 2008). 

In summary, the studies that investigated the effects of turnaround partner interventions using a compari- 
son group found mixed results for student achievement. Several studies used qualitative methods to identify 
conditions that differed between schools that improved after state intervention and schools that did not. 
Schools that improved had strong leadership, used data to guide instruction, had a positive school culture 
characterized by trust, and had increased expectations for students. 

Research on state interventions that did not invoive turnaround partners 

The literature search identified seven additional studies addressing other types of state interventions. These 
studies examined school improvement planning with additional funding, school restructuring, changes in 
the entity operating the school, and school closure. Studies of interventions that did not involve turn- 
around partners were few and addressed a disparate group of intervention strategies. It is challenging to 
draw any conclusions about the effectiveness of these strategies or the conditions under which they will be 
most likely to be successful based on this limited body of research. 

Studies on school improvement planning with additional funding 

Two studies (one of which was described in three related reports) examined school improvement planning 
with additional funding that came from two interventions specified by Massachusetts’s 2010 Act Relative to 
the Achievement Gap. Schools identified as the state’s persistently lowest performing schools were required 
to develop turnaround plans that addressed Massachusetts’s 11 Conditions for School Effectiveness (Lane, 
Unger, & Rhim, 2012).^ Some of the schools also received School Redesign Grants, with funding from the 
federal School Improvement Grant program, to support their improvement efforts. 

Lane et al. (2012, 2013) and Lane, Unger, and Souvanna (2014) examined 34 schools that developed turn- 
around plans — 30 of which also received School Redesign Grants — from 2010 to 2013 and compared 
schools that improved to those that did not. Four factors distinguished the two groups of schools over 
time: leadership, shared responsibility, and professional collaboration; intentional practices for improving 
instruction, including individualized support of teachers’ ability to identify student needs and deliver stu- 
dent-responsive instruction; providing student-specific instruction and supports to all students; and a safe, 
respectful, and collegial climate for teachers and students. Although schools’ success was attributed to these 
factors, other unidentified factors may explain the difference in outcomes. 
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LiCalsi, Citkowicz, Friedman, and Brown (2015) compared schools that received School Redesign Grants 
to schools in the same district that did not. The study found positive, statistically significant effects of the 
School Redesign Grants for student achievement in math and English language arts. Although differences 
between the two groups of schools before receiving the grants were controlled for, the study did not provide 
detailed information on how similar the two groups of schools were in student achievement before the 
intervention. The results should thus be interpreted with caution. 

Studies on school restructuring 

Two studies reported on school restructuring. 

Fields (2011) reported on three schools in Texas that had been restructured into theme-focused magnet 
schools in response to a state law that requires a school to restructure when it receives an academically 
unacceptable rating for two years. The schools were selected for study because they demonstrated growth 
in student achievement after they were restructured. Strong leadership and ongoing involvement of staff in 
the planning and implementation of the redesign were found to be present in the schools. 

Albarracin (1999) examined one low-performing school in New York that was required by the state to close 
and to open as a redesigned school the following year. School staff, parents, district staff, and state staff par- 
ticipated in a collaborative process to develop the plan for the redesigned school. The result involved divid- 
ing the school into two “houses,” one for grades K-3 and one for grades 4-6, instituting new approaches 
to school management, new curricula, and changes to the daily schedule. The school did not demonstrate 
growth in student achievement after reopening as a redesigned school. 

Studies on changes in the entity operating the school 

Two studies examined changes in the entity operating the school. 

Freshwater (2012) used case study methodology in four schools from one large urban school district that 
were required by their state to replace staff and work with an external partner to improve teaching and 
learning. Operation of two schools was turned over to an education management organization; in the other 
two schools, all staff were replaced. After three years of implementation all four schools were still per- 
sistently low performing, though some incremental improvement in student achievement had been made in 
three of them. The one school that did not show steady improvement experienced high principal turnover. 

Dworaczyk (2008) also used case study methodology to examine one school in California where the con- 
version to a charter school was followed by increased student achievement. The study identified a variety 
of changes made when the school converted to a charter that school staff felt contributed to the improve- 
ment, including better qualified staff, differentiated instruction, strong leadership, smaller school size, 
higher expectations, staff development, and enrichment activities for students. The results should be inter- 
preted with caution, as all conclusions were based solely on staff perceptions. 

Study on school closure 

One study examined school closure. Silander (2012) compared a treatment group of grade 6 students who 
enrolled in one of 14 New York middle schools that were subsequently closed to a comparison group of stu- 
dents who enrolled in grade 6 four years later who would have been likely to enroll in one of the 14 middle 
schools had they not closed. Although propensity score matching was used to create groups of students 
that were similar on assessment scores, attendance, and demographic characteristics, the method does not 
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account for other factors, such as political or societal factors, that may have changed over the four years 
that elapsed between the treatment and the comparison groups’ enrollment in middle school. The students 
in the comparison group demonstrated greater gains on assessments in English language arts and math 
than the students in the treatment group. 

In summary, substantially fewer studies examined interventions that did not involve turnaround partners. 
These studies examined school improvement planning with funding, school redesign, a change in the 
entity operating a school, and school closure. Only one or two studies of each type of intervention were 
located, which makes it challenging to draw any conclusions about the effectiveness of these strategies or 
the conditions under which they will be most likely to be successful. 

Limitations and impiications of the study 


This report summarizes the research on the association between state interventions in chronically low- 
performing schools and student achievement (see table 1). Although the study team searched for research 
on a wide variety of state interventions, the vast majority of the research that was located focused on one 
type of state intervention: working with a turnaround partner. Previous research has indicated that many 
states have policies to permit this type of state intervention. For example, 41 states have policies allowing 
the state or its designee to conduct an instructional audit, and 35 states have policies that permit the state 
to appoint a turnaround partner or technical assistance provider (Klute et al, in press). Despite the popm 
larity of this type of intervention across states, nearly half the studies in this review were conducted in just 
one state, California. 

A substantial limitation in the existing literature is that most studies used a research design that does not 
permit causal conclusions about the effects of these interventions. Less than a third of identified studies 
used a quashexperimental design that compared schools that received an intervention to schools that did 
not (see table 1). Many of these studies had serious limitations, including confounds created when only one 
school was assigned to each condition or when the treatment and control groups attended school at widely 
different points in time. Other studies did not provide the information needed to assess the extent to which 
treatment and comparison groups were similar at the start of intervention. 

Four studies compared schools that made improvements under the intervention to those that did not. 
These retrospective studies seek to identify factors or characteristics that distinguish more and less success- 
ful schools. Because the identified factors or characteristics were not identified and systematically assigned 
to schools at the start of the intervention, it is impossible to know whether those factors are truly the cause 
of the school’s success rather than other unmeasured factors. The remaining studies (about half) were 
descriptive studies of one school or a group of schools. While these studies provide rich detail about what 
took place in the schools experiencing state interventions, they rarely compare schools that receive an 
intervention to other schools. 

Retrospective and descriptive designs can be useful for identifying characteristics or strategies that could 
be addressed in future research. The studies summarized in this report identified strong leadership, use of 
data to guide instruction, positive school culture characterized by trust, and increased expectations for 
students as factors that seem to be associated with positive student outcomes in schools experiencing state 
intervention. Future research may need to use more rigorous designs that can estimate the impact of these 
interventions on student achievement in order to be of maximum benefit in guiding policymakers and state 
education agency staff as they design interventions in chronically low-performing schools. 
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Studies on state interventions rarely used research designs, such as quashexperimental designs, that can 
provide evidence about the impact of the intervention. Only one type of intervention, turnaround partner 
interventions, had more than one study that compared schools receiving the intervention to those not 
receiving it (see table 1), and all identified studies had methodological concerns that limit the strength 
of their conclusions. Among the six studies of turnaround partners that used a comparison group, one 
reported a positive effect of the intervention (Thompson et al, 2011), three reported nonsignificant results 
(Bitter et al, 2005; Goe, 2006; and Harr et al, 2007), and two reported mixed findings (Scroggins, 2012; 
Tapia, 2008). As a group, these studies do not provide consistent evidence for the effectiveness of this type 
of intervention. 

Very little research was located regarding interventions that did not involve a turnaround partner, such as 
school closure, charter conversion, and school redesign. Because only one or two studies of each of these 
interventions were located, it is difficult to draw conclusions about the effectiveness of these interventions 
or the conditions under which these interventions are most likely to be successful. Future research could 
examine implementation of these types of interventions in relation to student achievement. 
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Appendix A. Literature search procedures 


To identify what the research literature suggests about the interventions and implementation features 
associated with improving student achievement in chronically low-performing schools, the study team 
conducted a comprehensive search to identify relevant studies. The search proceeded in two steps: an 
electronic database keyword search and an intervention search. After the search, the study team consulted 
with content area experts to identify any other studies that were missing. 

Electronic database keyword search 

An electronic database keyword search was the first step in identifying research on state interventions 
in chronically low-performing schools. The study team used keywords to search four electronic databases 
during August-October 2014: ERIC, PsyclNFO, ProQuest Dissertations and Theses, and Academic Search 
Premier. The study team was trained to screen studies and keep those that met the following relevance 
criteria: 

• Must be a study or a literature review. 

• Must examine a state-initiated intervention in one or more chronically low-performing schools. 

• Must involve a K-12 public school in the United States, including traditional schools, charter 
schools, and magnet schools. 

• Must be published between January 1994 and October 2014. 

• Must be written in English. 

Search terms included 69 keywords related to state interventions that were developed after reviewing states’ 
Consolidated State Performance Reports for 2009-14 and the most current versions of each state’s Elemen- 
tary and Secondary Education Act Flexibility Requests as of August 2014. A consultant with expertise 
in interventions for chronically low-performing schools reviewed the keywords and provided suggestions 
for additional keywords. Final keywords are listed in box Al. Each keyword was combined with the words 
“state” and “school” during the searches to identify reports that included all three search terms. Keywords 
in quotation marks indicate those that the study team used to search for exact phrases (for example, “edu- 
cation management organization” was searched as one term in which all three words in sequence needed to 
be present), and asterisks indicate wildcards, used so that words with different endings could be found (for 
example, “govern*” would produce results that include the word “governing” and “governance”). 

Initial keyword searches returned 35,963 records (figure Al). The study team conducted the initial screen 
by reviewing study titles and abstracts to determine whether studies met the relevance criteria described 
above. When it was unclear whether a study met relevance criteria based on the information contained 
in the title and abstract, researchers included the study to be more closely reviewed in the second step of 
screening. The electronic database keyword search and initial screening identified 317 potentially relevant 
reports. 

One member of the study team conducted a second screening of the potentially relevant reports, consult- 
ing the full text of each document as necessary to determine whether the report was indeed a research 
study, defined as a report that presented qualitative or quantitative data to address a research question 
using clearly defined methods. Another member of the study team also screened a random selection of 
10 percent of the 317 reports to check the accuracy of the first study team member’s screening decisions. 
This check confirmed 88 percent of screening decisions. For the remaining 12 percent of decisions, study 
team members discussed the ways in which each report met or did not meet each criterion until they 
reached consensus about the final screening decision. After the second screening 139 studies remained. 
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Box Al. Keyword search terms 


accountabil* 

“improvement plan" 

“scholastic audit" 

“ASSiST team” 

ineffective 

“school administrative team” 

“assistance team” 

“instructional audit” 

“solutions team” 

chaiienge 

“leadership coach” 

“state appointed" 

“charter conver*” 

“low perform*” 

“state-appointed” 

closure 

“low-perform*” 

“state directed” 

conservatorship 

“memorandum of understanding” 

“state-directed” 

contract* 

MOU 

“state review panel” 

“decision making” 

“needs assessment” 

“state system of support" 

“decision-making” 

“new start” 

SSOS 

“direct management" 

non-complian* 

“structural change” 

discontinuance 

“operational control” 

struggling 

dissolve 

“parent trigger” 

takeover 

distress* 

privatize 

“technical assistance" 

“education management organization" 

“program improvement” 

“third party” 

EMO 

“public-private partner*” 

“third-party" 

“education speciaiist" 

receiver* 

turnaround 

“externai management” 

reconstitute* 

turn-around 

“externai review" 

recovery 

“turn around” 

faiiing 

“replace staff” 

unacceptable 

govern* 

restaffing 

underachieving 

“highly skilled educator” 

restart 

underperform* 

“implementation plan” 

restructuring 

unsatisfactory 


EMO is education management organization; MOU is memorandum of understanding; SSOS is statewide system of support. 
Note: Quotation marks indicate exact phrases; asterisks indicate wiidcards. 

Source: Authors' compiiation. 


Figure Al. The study team used five steps to screen reievant studies 


Initial keyword searching: 35,963 studies 


Initial title and abstract screen: 317 studies 



Source: Authors' construction. 
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Intervention search 


Based on the review of Consolidated State Performance Reports and Elementary and Secondary Education 
Act Elexibility Requests, the study team developed a list of state-specific interventions. A consultant with 
expertise in interventions in chronically low-performing schools reviewed the list and offered suggestions 
for additional interventions. The final list includes 122 specific interventions (box A2). The study team 
searched the same four databases used for the keyword search on these names. This process identified 50 
additional studies. 


Box A2. Interventions by state 
Alabama 

Continuous Improvement Plan 
Regional InservIce Centers 
Regional Planning Teams 
Regional Support Staff 
Turnaround Teams 

Arizona 

Continuous Improvement Plan 

Alaska 

Alaska Effective Schools 
Framework 

Alaska STEPP (Steps Toward 
Educational Progress and 
Partnership) 

Arkansas 

Priority Intervention Plan (PIP) 

California 

Intervention Teams 

Colorado 

Coordinated Support Teams 
External Providers 
Innovation School 
Performance Managers 
School Performance Framework 
State Review Panel 
Unified Improvement Plan* (UIP) 

Connecticut 

Commissioner’s Network 
Comprehensive Success Review 
Priority Zone 
Success Plan 


Florida 

Continuous Improvement Model 
Florida Turnaround Leaders Program 
Hybrid Model 
Instructional Review 
Intervene Option Plan 

Georgia 

Georgia Assessment of 
Performance on School 
Standards (GAPSS) 

Hawaii 

Office of School Transformation 
Priority Academic and Financial 
Plan 

Zones of School Innovation 

Idaho 

Capacity Builders 
Governance Partnership Model 
Instructional Core Focus Visit 
WISE (Ways to Improve School 
Effectiveness) 

Illinois 

Comprehensive Audit 

Indiana 

Intervention Plans 
Lead Partner 
Turnaround Academ* 

Turnaround School Operator (TSO) 

Kansas 

Action Plans 
Integrated Innovation 


Kentucky 

Consolidated School Improvement 
Plan 

Distinguished Educator 
Education Recovery 

Louisiana 

Academically Unacceptable 
Schools 

Baton Rouge Achievement Zone 
Distinguished Educator 
Network Teams 
Recovery School District 

Maine 

School Coaches 

Maryland 

Breakthrough Center 
School Improvement Planning 

Massachusetts 

Redesign Plan 
Tri-State Rubric 

Michigan 

District Intervention Team 
Education Achievement Authority 
Education Achievement System 
School Improvement Framework 

Minnesota 

Minnesota Common Principles of 
Effective Practice 
Regional Centers of Excellence 

(continued) 
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Box A2. Interventions by state (continued) 


Mississippi 

Implementation Specialists 
Mississippi SOARS (Schools 
Obtaining Academic Results for 
Success) 

School at Risk 

Missouri 

Comprehensive School 
Improvement Plan 
Regional School Improvement 
Teams 

Nevada 

Nevada Comprehensive Curriculum 
Audit Tool for Schools 
SAGE (Student Achievement Gap 
Elimination) 

New Hampshire 

State School Improvement Team 
Steps to Success 

New Jersey 

Quality School Review 
Regional Achievement Centers 

New Mexico 

Priority Schools Bureau 

New York 

Diagnostic Tool for School and 
District Effectiveness 
Distinguished Educator 
Educational Partnership 
Crganization (EPC) 

Metrics and Expectations 
School-Based Inquiry Teams 
Tri-State Rubric 

North Carolina 

Roundtables 

School Implementation Team 
Transformation Coach 

* indicates a wildcard. 

Source: Authors’ compilation 


Ohio 

Academic Distress Commission 
Diagnostic Review 
Dhio Improvement Process 
Parent Takeover Pilot Project 
Transformation Specialists 

Oklahoma 

C3 Schools 

Priority Schools Advisory Board 
WISE (Ways to Improve School 
Effectiveness) 

Oregon 

Comprehensive Achievement Plan 
Cycle of Improvement 
School Appraisal Teams 

Pennsylvania 

Academic Recovery Liaison 
Comprehensive Planning Tool 
Standards Aligned System 

Rhode Island 

Diagnostic Screen 
Flex Model 
Tri-State Rubric 

South Carolina 

Challenge to Achieve Plan 
External Review Process 
South Carolina Turnaround Leaders 
Program 

South Dakota 

School Support Team 
SD LEAP (Leading Effectively 
Achieving Progress) 

Tennessee 

Achievement Advisory Council 
Achievement School District 
Consortium on Research, 

Evaluation and Development 
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Innovation Zone 
School Support Team 

Texas 

Alternative Management 
Campus Intervention Team 
District Coordinator of School 
Improvement 

Professional Service Provider 
Repurposing 

Texas Title 1 Priority Schools 
(TTIPS) 

Texas Turnaround Leadership 
Academy 

Utah 

School Support Team 

Virginia 

Lead Turnaround Partner 
Cpportunity Education Institution 
Partnership for Achieving 
Successful Schools 
Project Graduation 
School Turnaround Specialist 
Program 

School-Level Academic Reviews 
Turnaround Zone 

Washington 

Academic Performance Audit 
Required Action Districts 

West Virginia 

Extended Strategic Plan 
School Improvement Coordinators 

Wisconsin 

Committee on District and School 
Improvement 
Reform Plan 
Turnaround Partner 


Expert consultation 


A list of all located reports was sent to two consultants with expertise in interventions in chronically 
low'performing schools. The study team asked them to review the list and to identify any potentially rele- 
vant studies that were not included. Seven additional reports were suggested, five of which met the screen- 
ing criteria. One of these reports was published in 2015, after the timeframe for the initial searches, but was 
included because it was relevant and met other screening criteria. 

Final screening 

During the final screening, three study team members carefully read each report and screened against 
three criteria. First, they assessed whether the intervention met the definition of state interventions, which 
include actions taken directly by states or their designees to improve chronically low-performing schools 
or actions that states required schools or districts to take to improve chronically low-performing schools. 
Second, they confirmed that the study examined student academic outcomes. Third, they assessed whether 
the report clearly stated research questions or objectives that linked the intervention to the student out- 
comes. After this final screening, 27 reports on the 25 studies described in this report remained. 
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Appendix B. Characteristics of studies inciuded in the iiterature review 


Table B1 presents details on each of the 25 studies, including the year the intervention was implemented, study 
location, sample size, and research methods used. 


Table Bl. Characteristics of the 25 studies included in the literature review 

study 

Intervention 

name 

state 

Year 

implemented 

Data source 

Comparison 

Grade level 

Number 
of schools 
included 
in study 

Interventions involving turnaround partners 

Bibian (2006) 

Joint 

intervention 

agreement 

California 

2002/03 

Interviews with 

school and district 

staff, classroom 
observations, stu- 
dent assessment 

data 

None 

Middle 

1 

Bitter et al. 

(2005) 

Immediate 

Intervention/ 

Underperforming 

Schools 

Program, school 

assistance and 

intervention 

team 

California 

2003/04 

Interviews with 

school and district 

staff, interviews 

with school 

assistance and 

intervention team 

leaders, student 

assessment data 

Compared partici- 
pating schools to 
similar nonpartici- 
pating schools 

Elementary, 
middle, and 
high 

2,338 

Cepela (2008) 

State System of 
Support 

Illinois 

2006/07 

Interviews with 

school and district 
staff, analysis of 
school improve- 
ment plan docu- 
ments, student 

assessment data 

None 

Middle 

2 

Cruz (2007) 

Mandated use 

of Malcolm 

Baldridge 

Education 

Criteria for 

Performance 

Excellence 

New Mexico 

2005/06 

Surveys of school 
staff, interviews 
with principals, 

student achieve- 
ment data 

None 

Elementary 

and middle 

4 

Davis (2007) 

Joint 

intervention 

agreement. 

Comprehensive 

School Reform 

program 

California 

2002/03 

Surveys of teach- 
ers and adminis- 
trators, review of 
documents, inter- 
view with the area 
superintendent, 

student assess- 
ment data 

None 

Middle 

1 

Dominguez, 
Nicholls, & 

Storandt 

(2006) 

External review 

team 

South Carolina 

2001/02 

Interviews with 

external review 

team members 

and school staff 

review of docu- 
ments, student 

assessment data 

Compared 

schools that 

improved to those 

that did not 

Elementary, 
middle, and 
high 

62 

(continued) 
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Table Bl. Characteristics of the 25 studies included in the literature review (continued) 


study 

Intervention 

name 

state 

Year 

implemented 

Data source 

Comparison 

Grade level 

Number 
of schools 
Included 

In study 

Goe (2006) 

immediate 

intervention/ 

Underperforming 

Schoois 

Program 

Caiifornia 

1999-2002 

Interviews with 

school staff, 

classroom obser- 
vation, student 

achievement data 

Compared schools 
that participated 

in the Immediate 

Intervention/ 
Underperforming 
Schools Program 

to schools that 
qualified for but 
did not apply to 
the program 

Elementary, 
middle, and 
high 

3,221 

Harr, Parrish, 
Socias, & 

Gubbins 

(2007) 

High Priority 

Schoois Grant 

Program 

Caiifornia 

2002-06 

Student assess- 
ment data, analy- 
sis of documents 

Compared schools 
that participat- 
ed in the High 
Priority Schools 
Grant Program to 
low-performing 

schools that did 

not participate 

Elementary, 
middle, and 
high 

319 

Jefferson 

(2006) 

Aiabama State 

Takeover Modei 

Alabama 

1999-2001 

Surveys of school 
staff, student 

assessment data 

None 

Elementary, 
middle, and 
high 

10 

Johnson 

(2008) 

Assistance team 

North Carolina 

2005/06 

Interviews with 

school staff, sur- 
veys of teachers, 

student assess- 
ment data 

None 

High school 

1 

Macias (2011) 

Technicai 

assistance team 

Texas 

2006-09 

Interviews and 
focus groups with 

school and district 

staff, student 

assessment data 

None 

Elementary 

1 

Scroggins 

(2012) 

Distinguished 

Educator 

Program 

Louisiana 

2008-10 

Student assess- 
ment data 

Compared distin- 
guished educator 

schools to those 

that did not 

receive a distin- 
guished educator 

Elementary 

288 

Sipp (2008) 

Collaborative 

Assessment 

and Planning for 

Achievement 

New Jersey 

2004-07 

Student assess- 
ment scores, 

administrator inter- 
views, surveys of 
teachers, analysis 

of documents 

None 

Elementary 

2 

Tapia (2008) 

Schooi 

Assistance and 

intervention 

Team 

California 

2005/06 

Surveys and inter- 
views with school 

staff, observa- 
tions, student 

assessment data 

Compared the 

school with a 

school assistance 

and intervention 

team to a similar 

school without a 

team 

Elementary 

2 








(continued) 
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Table Bl. Characteristics of the 25 studies included in the literature review (continued) 

study 

Intervention 

name 

State 

Year 

implemented 

Data source 

Comparison 

Grade ievei 

Number 
of schools 
included 
in study 

Timar & Chyu 
(2010) 

High Priority 
Schools Grant 

Program 

California 

2004-08 

Interviews and 
focus groups 
with school staff, 

classroom obser- 
vations, student 

assessment data 

Compared 
improving schools 
to nonimproving 

schools 

Elementary 
and high 

15 

Thompson, 
Brown, 
Townsend, 
Henry, & 
Fortner (2011) 

Turnaround 

schools 

North Carolina 

2006-10 

Student assess- 
ment data, inter- 
views with school 

and district staff 

and turnaround 

partners 

Compared turn- 
around schools 

to schools not 
receiving the inter- 
vention, compared 
improving schools 
to nonimproving 

schools 

Elementary, 
middle, and 
high 

212 

Wakelyn 

(2006) 

Distinguished 

Educator 

Program 

Kentucky 

1994-96 

Interviews with 

school staff, stu- 
dent assessment 

data 

None 

High school 

4 

Woods (2004) 

Immediate 

Intervention/ 

Underperforming 

Schools 

Program 

California 

1999-2002 

Survey of prin- 
cipals, student 

assessment data 

Compared suc- 
cessful schools 

to less successful 

schools 

Elementary 

107 

Interventions involving school improvement planning with additional funding 

Lane, Unger, 

& Rhim 

(2012, 2013); 
Lane, Unger, 

& Souvanna 

(2014) 

Conditions 

for School 

Effectiveness, 
School Redesign 

Grants 

Massachusetts 

2010-13 

Monitoring site 
visit reports. 

School Redesign 

Grant renewal 

applications, stu- 
dent assessment 

data 

Compared 

schools that 

improved to those 

that did not 

Elementary, 
middle, and 
high 

34 

LiCalsi, 
Citkowicz, 
Friedman, & 
Brown (2015) 

School Redesign 
Grants 

Massachusetts 

2010-13 

Student assess- 
ment data, atten- 
dance data 

Compared schools 
that received 

School Redesign 

Grants to other 

schools in the 

same districts 

Elementary, 
middle, and 
high 

31 schools 

that received 

School 

Redesign 

Grants; 

number of 

comparison 
schools is 

not specified 

Interventions involving school restructuring 

Albarracin 

(1999) 

School redesign 

New York 

1996/97 

Student assess- 
ment data; doc- 
ument analysis; 

unstructured inter- 
views with school 

staff and parents; 
surveys of teach- 
ers, administra- 
tors, parents, 
and students; 

observations 

None 

Elementary 

1 

(continued) 
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Table Bl. 

Characteristics of the 25 studies included in the literature review (continued) 


study 

Intervention 

name 

State 

Year 

implemented 

Data source 

Comparison 

Grade level 

Number 
of schools 
included 
in study 

Fields (2011) Reconstitution 
into a theme- 

focused magnet 

school 

Texas 

2009/10 

Interviews with 

school and district 

staff, surveys of 
teachers, analysis 

of documents 

None 

High school 

3 

Interventions involving changes in the entity operating the school 

Dworaczyk 

(2008) 

Convert to 

charter 

California 

2005/06 

Surveys and inter- 
views with school 

staff, classroom 

observations 

None 

Elementary 

1 

Freshwater 

(2012) 

District 

reconstitution, 

education 

management 

organization 

Not stated 

2007-10 

Interviews with 

school, district, 

and education 

management 
organization 
staff; observa- 
tions; analysis of 

documents 

None 

High school 

4 

Interventions involving school closure 

Silander 

(2012) 

School closure 

New York 

2006-08 

Student assess- 
ment scores, 

attendance data 

Compared 

students in the 

school that sub- 
sequently closed 

to students who 

attended an 

alternative school 

after school 

closure was 

announced 

Middle 

14 

Source: Authors' compilation. 
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Notes 


1. Throughout this report “state education agency” refers to the state-level government education agency 
or department of education responsible for supervising public elementary and secondary education. 

2. Goe (2006) compared participating schools to all schools that qualified, regardless of whether they 
applied to the program. Bitter et al. (2005) compared the first cohort of participating schools to a com- 
parison group of schools that had applied for the program but had not been selected; for the second and 
third cohorts of participating schools, all eligible schools were used as the comparison group because 
most of the schools that applied were selected. 

3. Successful schools were defined as those that gained at least 90 points on the Academic Performance 
Index between 1999 and 2002. Less successful schools were defined as those that gained 30 points or 
less between 1999 and 2002. Academic Performance Index scores are based on the national percentile 
rank scores for students in each school on the state assessment (the Stanford Achievement Test, Ninth 
Edition) and range from 200 to 1,000. 

4. Schools that improved were defined as schools that met targets for adequate yearly progress and growth 
on the Academic Performance Index for two years or more during the funding period. Schools that did 
not improve were defined as schools that did not meet targets for adequate yearly progress or growth on 
the Academic Performance Index for three years. 

5. The scholastic audit is a five-day process during which audit team members collect information about 
the school through interviews, classroom observations, document review, surveys, meetings, and focus 
groups with school staff, students, and parents (Bibian, 2006; Davis, 2007). 

6. While the findings seem contradictory, this pattern of results may be due to the fact that the Aca- 
demic Performance Index is sensitive to improvement across the range of performance. For instance, 
a student moving from “far below basic” to “basic” would contribute to a change in the Academic 
Performance Index score. However, this same student would be considered “not proficient” at both time 
points. Similarly, a student moving from “proficient” to “advanced” would contribute to the change in 
the score but would be considered “proficient or advanced” at both time points. 

7. Criteria that were used for identifying comparison schools varied depending on the grades the schools 
served. At the high school level, comparison schools had a performance composite below 66 percent 
in 2004/05 or 2005/06. At the middle school level, comparison schools had a performance composite 
below 75 percent in 2004/05 or below 55 percent in 2005/06. 

8. The state system of support in Illinois and the Collaborative Assessment and Planning for Achieve- 
ment initiative in New Jersey were developed as ways to meet the mandates of No Child Left Behind 
(Cepela, 2008; Sipp, 2008). The Texas Education Code requires that any school that is rated “aca- 
demically unacceptable” be assigned a technical assistance team (Macias, 2011). In 1996 the North 
Carolina legislature passed legislation that required low-performing schools to be assigned assistance 
teams (Johnson, 2008). 

9. The 11 conditions are effective district systems for school support and intervention; effective school 
leadership; aligned curriculum; effective instruction; student assessment; principal’s staffing authority; 
professional development and structures for collaboration; tiered instruction and adequate learning 
time; students’ social, emotional, and health needs; family-school engagement; and strategic use of 
resources and adequate budget authority. 
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